Simultaneous Recurrent Neural Networks for Static Optimization

This paper presents a study on computational promise of Simultaneous Recurrent Networks to solve large-scale optimization problems. Specifically the performance of the network for solving Traveling Salesman Problem is addressed and analyzed. A recurrent and trainable neural network, Simultaneous Recurrent Network, with Recurrent Backpropagation training algorithm is employed to address difficulties related to scaling problem, which is currently hindering the successful application of Artificial Neural Network algorithms to large-scale static optimization problems. The Simultaneous Recurrent Neural Network was successfully trained to locate "good quality" solutions to the Travelling Salesman Problem with up to 500 cities. INTRODUCTION Artificial Neural Networks (ANN) are of great interest on the part of government, industry and academia on the potential contribution that this new information processing technology can make in everyday life. As the complexity of a problem increases the relative improvement in time to find the acceptable solution, gained by ANN technology over conventional technology becomes more pronounced. Conventional approaches tend to perform poorly on complex problems, even though for simple problems they appear to perform satisfactory. ANNs offer a very attractive choice for efficiently searching, in real-time, the very large search space for optimization problems of real-life complexity. The Hopfield network (HN) and its derivatives including the Boltzmann Machine and the Mean Field Annealing network seem to be most prominent and extensively applied ANN algorithms to solve static optimization problems [Hopfield et.al., 1985; Smith et al., 1998]. But, HN and its derivatives do not scale well with increase in the size of the optimization problem. The main research goal is to address and solve the scaling problem ANN algorithms currently experience for static optimization problems. Towards that goal, we will explore and assess the computational promise of the Simultaneous Recurrent Neural Network [Pang and Werbos, 1997], a trainable and recurrent ANN algorithm, for static optimization problems. Existing neural optimizer algorithms are either trainable feedforward architectures or recurrent architectures with preprogrammed weight structures. SIMULTANEOUS RECURRENT NEURAL NETWORK A simultaneous recurrent network (SRN) is an Artificial Neural Network [Pang and Werbos, 1997], which has external inputs in the form of a vector X, a feedforward function ƒ(.) (any feedforward network including multi-layer perceptron is appropriate), outputs in the form of vector Y and a feedback path, which copies the outputs to inputs without a time delay. A formal description of SRN is formulated in [Werbos, 1992] who defines an SRN as a mapping